TOJET: The Turkish Online Journal of Educational Technology - October 2012, volume 11 Issue 4 


COMPARISONS OF LEARNER-GENERATED VERSUS INSTRUCTOR-PROVIDED 

MULTIMEDIA ANNOTATIONS 

Dr. Chiu-Jung Chen 
National Chia-Yi University, Taiwan 
No.300 Syuefu Rd., Chiayi City 60004, Taiwan (R.O.C.) 

Email: chenc@mail.ncyu.edu.tw 

Dr. Pei-Lin Liu* (corresponding author) 

National Chia-Yi University, Taiwan 
No.300 Syuefu Rd., Chiayi City 60004, Taiwan (R.O.C.) 

E-mail: peilin@mail.ncyu.edu.tw 


ABSTRACT 

The purpose of this study was to explore the effectiveness of using learner-generated and instructor-provided 
multimedia annotations on foreign language reading comprehension and attitudes. The four research questions 
are: (1) what are the effects of using different multimedia annotations on reading comprehension for learners of 
different cognitive learning styles (field-dependent and field-independent)? (2) What are the effects of using 
different multimedia annotations on reading comprehension for learners with different learning abilities (higher- 
level and lower-level)? And (3) what are learners’ attitudes toward using a multimedia annotation system? The 
results of this study are listed as follows: First, for reading comprehension, the learner-generated annotation 
group performed better than the instructor-provided group, no matter which cognitive learning style they were. 
Second, higher-level learners with learner-generated annotation performed better than those with instructor- 
provided annotation. Elowever, the difference between lower-level learners with learner-generated annotation 
and those with instructor-provided annotation was not significant. Finally, learners had positive attitudes toward 
multimedia annotation use and thought text annotation was the most useful of the different types. 

INTRODUCTION 

Annotation refers a note that is made while reading any form of text (Chen, Hwang, & Wang, 2012). The 
behavior of making marks on reading material is important in traditional learning activities (Hwang, Wang, & 
Sharpies, 2005). There is a great deal of research showing that annotation can facilitate reading comprehension 
(Hwang, Wang, & Sharpies, 2007; Pan, 2006; Sung, 2007), including that of second language (L2) learners 
(Widdowson, 1984). Roby (1999) divided annotation into two types: learner-generated and instructor-provided. 
Learner-generated annotation refers to annotations that are made by the learners themselves when they want to 
comprehend the reading texts. It can help remind them of the content or the significant points of the reading 
materials. However, learners may miss some important information or make mistakes while making annotations 
by themselves. 

Instructor-provided annotation is made by experts or instructors. It provides an efficient way for learners to 
acquire unfamiliar but important knowledge more easily. The advantage of instructor-provided annotation is that 
it provides more comprehensive and more correct information about the texts. It can also help less-able learners 
perform better. The disadvantage of instructor-provided annotation is that the annotation provided by the 
professional may not induce learners’ interest (Ariew & Ercetin, 2004; Akbulut, 2007a; Akbulut, 2007b; 
Akbulut, 2008; Chun, 2001; Saker & Ercetin, 2005). 

With the rapid development of technology, language learners have many opportunities to receive online reading 
information as hypertext with multimedia via the Internet (Ariew & Ercetin, 2004). As a result, how readers can 
annotate online reading materials has attracted considerable interest. Previous researchers have attempted to 
provide multimedia annotation tools which enable readers to make marks on online reading materials. 
Multimedia annotation refers to computer-based applications that provide annotation through multiple types of 
hypermedia, such as text, audio, video, graphics, and animation (Sakar & Ercetin, 2005). 

In terms of reading comprehension, there are many factors that influence learners’ reading comprehension, 
including annotation types (Ariew & Ercetin, 2004; Akbulut, 2007a Saker & Ercetin, 2005; Akbulut, 2008), 
cognitive learning styles (Akbulut, 2007a; Akbulut, 2008; Salmani-Nodoushan, 2005; Vivaldo-Lima, 1997), and 
learners’ reading proficiency ability (Ariew & Ercetin, 2004; Akbulut, 2007b). 

For annotation types, some studies have shown that learner-generated multimedia annotation (Hwang, Wang & 
Sharpies, 2007) or instructor-provided multimedia annotation (Lomicka, 1998; Sakar & Ercetin, 2005) have a 
positive influence on facilitating reading comprehension. Vivaldo-Lima (1997) pointed out that there was a 
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positive and highly important correlation between cognitive learning styles and reading comprehension. 
According to Salmani-Nodoushan (2005), cognitive learning styles can be divided into field-independence (FI) 
and field-dependence (FD). However, the result of Hwang, Wang and Sharpies’s (2007) study showed that there 
was no relationship between the quantity of annotation and the cognitive learning styles in multimedia reading 
environments. Proficiency level is also one of the predicators of reading comprehension in multimedia 
annotation learning environments (Ariew & Ercetin, 2004; Akbulut, 2007b). Hwang, Wang and Sharpies (2007) 
pointed out that the students with higher-level ability made more annotations. However, Chun’s (2001) study 
showed that although the higher-level students looked up fewer words than the lower-level students, the 
differences between the higher- and lower-level students were not significant. 

To sum up, the studies which are available have revealed insufficient and inconclusive results regarding what 
types of multimedia annotations learners with different cognitive learning styles prefer to use, and what types of 
multimedia annotations facilitate reading comprehension. Thus, the purpose of the present study was to explore 
the effectiveness of using different multimedia annotations (learner-generated vs. instructor-provided) on 
English reading among different characteristics of learners (cognitive learning styles and reading proficiency 
levels). The specific research questions were as follows: 

Q1 What are the effects of using different multimedia annotations on reading comprehension for 
learners of different cognitive learning styles? 

Q2 What are the effects of using different multimedia annotations on reading comprehension for learners 
with different learning abilities? 

Q3 What are learners’ attitudes toward using a multimedia annotation system? 

RESEARCH BACKGROUND 
Reading and annotation 

According to Brown (2007), reading ability will be developed best in association with appropriate reading 
strategies. Led by Goodman’s (1970) work, bottom-up and top-down processing became two distinctive reading 
methodologies. In terms of bottom-up processing, readers must first recognize linguistic signals to decode the 
texts. The linguistic signals include letters, morphemes, syllables, words, phrases, grammatical cues, and 
discourse markers. On the other hand, top-down processing refers to the fact that readers draw on their 
intelligence and experiences to understand texts. 

More recent research on reading has shown that a combination of bottom-up and top-down processing, what has 
been called the interactive reading model, is more appropriate (Akbulut, 2008; Chun & Plass, 1997). “This 
model takes into account the contribution of both text-driving features (i.e., decoding of the text) and reader¬ 
driving features (i.e., interpretation based on background knowledge). In such an approach, a weakness in one 
area can be compensated for in another area in successful reading” (Akbulut, 2008, p.40). 

Multimedia annotation offers various types of annotation, such as text, sound, graphics, video, and animation 
(Chun, 2001), and one major feature is the fact that it is nonlinear (Yu, Pedrinaci, Dietze, & Domingue, 2012). 
This means that the information units are networked, and can be processed in various orders. These features 
conform to the features of bottom-up processing, top-down processing, and the interactive reading model (Yang, 
Zhang, Su, & Tsai, 2011). Thus, they facilitate the comprehension of the texts simultaneously. 

From traditional annotation to multimedia annotation 

Traditional annotation means marks made by readers or professionals on reading matters (Hwang, Wang, & 
Sharpies, 2007), which can be presented before or during reading (Hwang, Shadiev, & Huang, 2011). The major 
functions of traditional annotation are to provide textual information, such as definitions of words, and to offer 
extra information related to the topic of the text (Ariew & Ercetin, 2004). Because of the rapid development of 
the Internet and technology, more kinds of multimedia are emerging in traditional annotation (Chen, Hwang, & 
Wang, 2012; Su, Yang, Hwang, & Zhang, 2010). According to Akbulut (2008), multimedia annotation refers to 
computer-based applications that provide information in a nonlinear way through multiple types of annotation. 

Regarding the comparison between traditional and multimedia annotation, some studies have indicated that 
multimedia annotation is more efficient than traditional annotation in terms of increasing learners’ vocabulary 
size (Roby, 1991;Kim & Kim, 2012), and reading comprehension (Hwang, Wang, & Sharpies, 2007; Lomicka, 
1998). Chen (2009), Lomicka (1998) and Hwang, Wang, and Sharpies (2007) showed that computerized reading 
with multimedia annotation promotes a deeper level of reading comprehension than no annotation or traditional 
annotation which provides only word definitions. Moreover, Lomicka (1998) claimed that those who had access 
to multimedia annotation generated a greater degree of ability in which learners connect events in a text at a local 
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or global level, and thereby demonstrate comprehension. The results of Chen’s (2009) study indicated that 
students using video annotation perform significantly better on tests than those using picture annotation. 

Multimedia annotation has a positive impact on L2 learning and teaching due to its integration of various media 
(Hwang, Wang, & Sharpies, 2007; Pan, 2006; Sung, 2007). According to Sung (2007), multimedia annotation 
systems have made a great contribution to language learning. First of all, learners have great motivation while 
using them. They provide learners with an opportunity to use a microphone to record their audio annotation for 
the content of the learning materials. Such systems also let them make text annotations that can help them review 
the information they have learned. Moreover, the learners can post some graphics that they think can remind 
them of the important information. 


Theoretical frameworks for instructor-provided and learner-generated annotation 


Theory 

Cognitive load 
theory 

Activity theory 


Table 1: Comparison of Two Theories of Multimedia Annotation 


Instructor-provided annotation 
Providing annotation decreases the level of 
extraneous cognitive processing and allows 
generative processing. 

Providing annotation does not encourage 
generative processing. 


Learner-generated annotation 
Generating annotation increases the level 
of extraneous cognitive processing and 
limits generative processing. 

Generating annotation by learners 
encourages generative processing. 


Table 1 summarizes the rationale for instructor-provided annotation on the basis of cognitive load theory and the 
rationale for learner-generated annotation on the basis of activity theory (adopted from Stull & Mayer, 2007, 
p.810). According to cognitive load theory, instructor-provided annotation can encourage learners to engage in 
generative processing by challenging them to see how the linear text was annotated using different modes of 
annotation, such as text, graphics, and so on, through organizing it into a coherent structure (Plass, Moreno, 
Briinken, 2010). Thus, learners are less likely to waste cognitive capacity on extraneous processing. Not asking 
learners to generate annotation does not necessarily encourage generative processing, but rather frees cognitive 
capacity that can be used for generative processing. In contrast, learner-generated annotation creates extraneous 
cognitive processing in which learners must focus on how to create annotation themselves, so leaving less 
capacity for generative processing (Stull & Mayer, 2007). This is the cognitive load theory rationale for learning 
by viewing. 

As for the activity theory rationale for learning by doing, it is based on the claim that learning occurs when 
learners are encouraged to engage in productive learning activity (de Jong, 2005; Kirschner, Sweller, & Clark, 
2006; Klahr & Nigam, 2004; Lillard, 2005; Mayer, 2003, 2004). Constructing annotations by learners can be 
considered a productive learning activity because the learner must engage in an activity that is related to the 
instructional objective—making relevant annotations on the text and organizing them into a coherent structure. 
According to activity theory, learner-generated annotation encourages generative processing, whereas instructor- 
provided annotation does not. Activity theory predicts that learners who read a text in which they generate their 
own annotation will perform better on reading quizzes than those who read the text with instructor-provided 
annotation (Stull & Mayer, 2007). Even though learning by doing appears to be an active treatment, it can inhibit 
generative cognitive processing if learners become confused as to how to carry out the task. The complexity of 
the activity may create extraneous cognitive processing which reduces the amount of cognitive capacity 
available for generative processing. 

METHOD 

Participants 

The participants were 93 students enrolled in two different sections of an English course at a mid-sized 
university in Taiwan. The two annotation treatments (learner-generated vs. instructor-provided) were each to be 
assigned to one intact class, but participants had already self selected the class at the beginning of the semester 
according to their preferences and individual needs. 

In order to understand the participants’ reading proficiency level, they were required to complete an English 
proficiency reading pre-test. And then, according to the scores of the test, those whose scores were lower than 
the average were classified as lower-level learners. Those whose scores were higher than the average were 
classified as higher-level learners. Participants were also asked to complete a Group Embedded Figures Test 
(GEFT) to identify whether their cognitive learning styles were field-independent (FI) or field-dependent (FD). 
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Instruments 
Reading material 

Three articles included in the high-intermediate proficiency level of an English magazine were selected as the 
reading materials in this research. The basic vocabulary size required for the readers was 3,000-5,000 words. The 
magazine was chosen as the teaching material not only in consideration of the articles being novel and lively, but 
also because they are graded to different English proficiency levels. It was not necessary for the learners to have 
any pre-knowledge of any specific domain before they could understand the articles. 

Pre and post English proficiency tests 

The purpose of the pre- and post-tests was to evaluate the subjects’ English ability before and after they read the 
articles. There were 25 reading comprehension test items for each test. Participants whose pre-test scores were 
higher than the average scores were regarded as higher-level learners. However, those whose scores were lower 
than the average were regarded as lower-level learners. The reliability of the pre-test was .80. The difficulty level 
was .56 for the pre-test and .55 for the post-test. 

The comprehension questions were meant to test learners’ understanding of the implications, meanings, and 
structures presented. Four types of questions included in the proficiency tests were: (1) factual questions, (2) 
inference questions, (3) main idea questions, and (4) tone questions. Factual questions are empirical questions- 
how/why things occur? Unlike factual questions, inference questions do not test learners’ knowledge of 
explicitly-cited facts, but rather their ability to draw conclusions from other information. Main idea questions ask 
the test taker to identify the passage's overall theme, as opposed to supporting facts and arguments. Tone 
questions usually ask what the author’s tone is, but may occasionally ask for the author’s attitude. 

Group embedded figures test (GEFT) 

The original GEFT test designed by WitKin et al. (1971) was utilized to measure the students’ FI/FD cognitive 
learning styles in this study (Chen, 2005, p.42). The number of simple figures correctly identified on the last two 
sections constituted the scores ranging from 1 (strongly field dependent) to 18 (strongly field independent). 
Those whose scores were higher than the average scores of all participants were assigned as having an FI 
cognitive style, while those whose scores were lower than the average were assigned as having an FD cognitive 
style. 

HyperMedia Editor (HME) annotation software 

HyperMedia Editor (HME) is free annotation software designed by Thibeault (2001). Learners can not only use 
this software to make text annotations, but can also use it to make graphics, audio, video, and web-link 
annotations. These functions of HME make it easier for learners to review the content they have read, and also 
help learners to become familiar with the habit of using multimedia-based learning material. The participants 
received two points for making one annotation. 





Have you ever wanted to learn about 
the complex life of ants or pondered 
the wonders of aviation and space 
travel? Or, maybe you find yourself 
Intrigued by African art or ancient 
human civilisation and culture. Believe 
It or not, you can investigate all of 
these things and more by visiting a 
Smithsonian museum. 


Beginnings 

The year was 1826 and lames 
Smithson, a British scientist, named 
his nephew as beneficiary In his will. 
According to the will, if the nephew 
should die without heirs, the estate 
would go to the United States of 
America. When the situation played 
out precisely this way years later, the 
Smithsonian Institution was created. 
Smithson specified that the 


Translation Glossary L*iks 


Figure 1 : A screen shot showing the interface of HME annotation (adopted from Liu, Chuang, Chen, & Yang, 

2010) 
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Annotation assignments and rubric 

The participants in the learner-generated group had to use HME to make multimedia annotations for the three 
articles as annotation assignments, and then turned in the assignments in class. The highest possible score for 
each type of annotation (text, graphics, audio, video, and web-link) was 20 points, and the total score of the 
assignment was 100. The rubric for the annotation assignment is as below (adopted from Yang, 2010, p. 44): 


Table 2: The Rubric of Annotation Scores 


Annotation type 

points 

Description 

Text Annotation 

20 

Make one text annotation, get 2 points (make over 10 text annotations, get 
20 points) 

Graphic Annotation 

20 

Make one graphic annotation, get 2 points (make over 10 graphic 
annotations, get 20 points) 

Audio Annotation 

20 

Make one audio annotation, get 3 points (make over 7 audio annotations, 
get 20 points) 

Video Annotation 

20 

Make one video annotation, get 10% (make over 2 video annotations, get 
20 points) 

Web-link Annotation 

20 

Make one web-link annotation, get 4 points (make over 5 web-link 
annotations, get 20 points) 

Total 

100 



Annotation attitude questionnaire 

The questionnaire was revised from Pan’s (2006) study, and included 40 items using a Likert five-point format 
(Strongly Agree, Agree, Undecided, Disagree, Strongly Disagree) and five personal information items. The main 
purpose of this questionnaire was to examine the learners’ perceived usefulness (N= 15), perceived ease of use 
(N= 10), and learning satisfaction (N= 15) regarding five functions of HyperMedia Editor (HME), namely text, 
graphic, audio, video, and web-link annotation, after the experiment. 

This paper utilized the SPSS software package to analyze the reliability of the questionnaire. The results are 
shown in Table 3. From the results, Cronbach alpha values in all dimensions are higher than .70 (total = .98). 
This shows the reliability of the questionnaire is sufficiently high. 


Table 3: Questionnaire dimension and the Cronbach alpha values 


# 

Dimension 

Cronbach alpha value 

1 

Perceived usefulness of HME 

.95 

2 

Perceived ease of use of HME 

.94 

3 

Learning satisfaction 

.96 

Total Cronbach alpha value 


.98 


Procedure 

The experiment lasted for eight weeks. The English reading proficiency pre-test was administered in the first 
week to know participants’ reading proficiency (higher-level vs. lower-level), and the result also confirmed that 
there was no significant difference in the pre-test for the two classes (t = - .83, p = .41). In the second week, the 
two classes were randomly assigned to the learner-generated annotation group and the instructor-provided 
annotation group. Both groups were asked to complete the Group Embedded Figures Test (GEFT) to know their 
cognitive learning styles (FI/FD). The third week took place in the computer lab and lasted two hours. After a 
one hour demonstration on how to operate the HyperMedia Editor (HME) annotation software, all participants 
were asked to practice using HME, and the learner-generated annotation group needed to practice how to make 
multimedia annotations for texts. 

From the fourth to six weeks, the participants in the learner-generated annotation group were asked to read one 
article each week, use HME to make annotations, and then turn in their annotation assignment in class. For the 
instructor-provided annotation group, participants were asked to use HME to read one article each week in class 
for general comprehension with annotations provided by the researcher. 

In the seventh week, a post-test was conducted in class to know participants reading comprehension of these 
three articles. The post-test took 20-30 minutes to complete. In the last week, all participants completed the 
Annotation Attitude Questionnaire to know their attitudes toward HME. Data were reported as means, ranks, and 
percentage of Likert scales for each annotation function. 
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RESULTS 

Learner-generated annotation had positive effects on both cognitive style learners 

To answer RQ1 - What are the effects of using different multimedia annotations (learner-generated vs. 
instructor-provided) on reading comprehension for learners of different cognitive learning styles (FI vs. FD)? 
The means and standard deviations for the different multimedia annotations of the learners of the different 
cognitive learning styles on the reading post-test are reported in Table 2. 

Table 4: Descriptive Statistics of Participants’ Reading Achievements by Annotation Type and Learning Style 


Learning style 

Annotation type 

N 

M 

SD 

Field-independent 

Learner-generated 

27 

67.41 

15.68 


Instructor-provided 

30 

44.40 

18.95 


Total 

57 

55.91 

17.32 

Field-dependent 

Learner-generated 

17 

62.35 

20.15 


Instructor-provided 

19 

38.53 

16.77 


Total 

36 

50.44 

18.46 


Two-way ANOVA was used to analyze the data to identify the interaction between using different multimedia 
annotations on reading comprehension for different cognitive learning style learners. Overall, the results revealed 
that learners with different learning styles had no significant difference in their reading achievements no matter 
what type of annotation they used (F = .23, p = .63). However, there is a significant difference between learners’ 
reading achievements by annotation type (F = 50.18, p = .00*). The learner-generated annotation group 
perfonned better than the instructor-provided annotation group no matter which cognitive style the learners were. 


Table 5: Results of Two-way ANOVA Analysis of Using Different Multimedia Annotation Types on Reading 
Comprehension for Learners of Different Learning Styles 


Source 

SS 

df 

MS 

F 

P 

Annotation Type 

9969.52 

1 

9969.52 

50.18 

.00* 

Learning Style 

53.88 

1 

53.88 

.27 

.60 

Annotation Type * Learning Style 

45.87 

1 

45.87 

.23 

.63 

Error 

17681.79 

89 

198.67 



Total 

269695.49 

93 





*p < .05 


Higher-level learners benefited more from learner-generated annotations 

To answer RQ2 - What are the effects of using different multimedia annotations (learner-generated vs. 
instructor-provided) on reading comprehension for learners with different proficiency levels (higher-level vs. 
lower-level)? The means and standard deviations of the different proficiency level learners’ performance on the 
annotation assignments by multimedia annotation type are reported in Table 5. The higher-level learners 
performed better than the lower-level learners no matter which multimedia annotation type they used. 

T able 6: Descriptive Statistics of Participants' Reading Achievement by Annotation Type and Proficiency Level 


Ability 

Annotation type 

N 

M 

SD 

Lower-level 

Learner-generated 

37 

41.71 

18.02 


Instructor-provided 

35 

38.67 

20.69 


Total 

42 

40.19 

19.34 

Higher-level 

Learner-generated 

37 

69.95 

13.45 


Instructor-provided 

14 

48.00 

11.64 


Total 

51 

58.98 

12.55 


Two-way ANOVA analysis revealed that there was a significant interaction between the learners with different 
annotation types and proficiency levels (F = 4. 79, p = .03*) (see Table 6). Learners with different proficiency 
levels had significant differences in their reading achievement when they used different types of annotation. The 
main effects of annotation type (F = 12.09, p = .00*) and proficiency level (F = 23.45, p = .00*) are also 
significant. By running the following two independent t-tests for learners with different proficiency levels, the 
results indicated that learner-generated annotation has more influence than instructor-provided annotation for 
higher-level learners on reading comprehension (t = 5.38, p = .00). However, there was no significant difference 
found for the lower-level learners. 
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Table 7: Results of Two-way ANOVA Analysis of Using Different Multimedia Annotation Types on Reading 
Comprehension for Learners of Different Proficiency Levels 


Source 

SS 

df 

MS 

F 

P 

Annotation Type 

1895.70 

1 

1895.70 

12.09 

.00* 

Ability 

3684.35 

1 

750.34 

23.45 

.00* 

Annotation Type * Ability 

750.34 

1 

156.82 

4.79 

.03* 

Total 

269695.49 93 




*p < .05 






Table 8: Results of Independent Samples T-test: learning achievements of lower level 


Annotation tvpe 

N 

M 

SD 

t 

P 

Lower-level Learner-generated 

37 

41.71 

18.02 

.36 

.72 

Instructor-provided 

36 

38.67 

20.69 



Higher-level Learner-generated 

37 

69.95 

13.45 

5.38 

.00* 

Instructor-provided 

14 

48.00 

11.64 




*p < .05 


Participants favored text annotation than others 

Regarding the perceived usefulness part, the main purpose of the questionnaire was to explore participants’ 
attitudes toward the usefulness of HME, including the text, graphic, audio, video, and web-link annotations. The 
results show that the participants thought the “text” (M= 3.36), “audio” (M= 3.35), and “graphic” (M=3.33) 
annotations made their reading more effective during the online reading process. As for their reading 
performance, the participants thought the “text” (M= 3.50), “video” (M =3.44), and “graphic” (M=3.42) 
annotations did improve it. 


Table 9: Analysis of Perceived Usefulness of HME 


Annotation 

M 

Rank 

SD 

D 

U 

A 

SA 

Function 



1 

2 

3 

4 

5 

Q (1.4.7.10.13): 

I think the annotation makes reading online become more effective. 



Text 

3.36 

1 

3.5% 

8.1% 

41.9% 

41.9% 

4.7% 

Graphic 

3.33 

3 

1.2% 

10.5% 

47.7% 

36.0% 

4.7% 

Audio 

3.35 

2 

2.3% 

12.8% 

41.9% 

33.7% 

9.3% 

Video 

3.23 

4 

0.0% 

15.1% 

51.2% 

29.1% 

4.7% 

Web-link 

3.17 

5 

1.2% 

19.8% 

45.3% 

27.9% 

5.8% 

Q (2.5.8.11.14): 

I think the annotation was useful during my online reading process. 



Text 

3.49 

1 

2.3% 

8.1% 

36.0% 

45.3% 

8.1% 

Graphic 

3.40 

2 

2.3% 

8.1% 

44.2% 

38.4% 

7.0% 

Audio 

3.40 

2 

2.3% 

14.0% 

32.6% 

43.0% 

8.1% 

Video 

3.40 

2 

0.0% 

11.6% 

44.2% 

37.2% 

7.0% 

Web-link 

3.23 

3 

1.2% 

16.3% 

44.2% 

34.9% 

3.5% 

Q (3.6.9.12.15): I think the annotation did improve my online reading performance. 



Text 

3.50 

1 

1.2% 

8.1% 

36.0% 

48.8% 

5.8% 

Graphic 

3.42 

3 

1.2% 

8.1% 

44.2% 

40.7% 

5.8% 

Audio 

3.36 

4 

2.3% 

12.8% 

39.5% 

37.2% 

8.1% 

Video 

3.44 

2 

0.0% 

8.1% 

45.3% 

40.7% 

5.8% 

Web-link 

3.26 

5 

2.3% 

15.1% 

45.3% 

29.1% 

8.1% 


Note. SA: strongly agree; A: agree; U: undecided; D: disagree; SD: strongly disagree. 


Regarding the results of perceived ease of use of annotation, most of the participants considered that “text”, 
“graphic”, and “video” can all be used without training and are easy to use (M > 3.40). 

_ Table 10: Analysis of Perceived Ease ofUse of EiME _ 

Annotation M Rank SD D U A SA 

Function 1 2 3 4 5 


Q (16.18.20.22.24): I think people can use the annotation without training. 

Text 3.45 1 3.5% 12.8% 30.2% 41.9% 11.6% 

Graphic 3.43 2 1.2% 10.5% 40.7% 39.5% 8.1% 
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Audio 

3.35 

4 

2.3% 

18.6% 

30.2% 

39.5% 

9.3% 

Video 

3.40 

3 

0.0% 

14.0% 

40.7% 

37.2% 

8.1% 

Web-link 

3.31 

5 

1.2% 

18.6% 

36.0% 

36.0% 

8.1% 

Q (17.19.21.23.25): I think the annotation was easy to use. 




Text 

3.44 

2 

5.8% 

10.5% 

30.2% 

40.7% 

12.8% 

Graphic 

3.49 

1 

1.2% 

8.1% 

41.9% 

38.4% 

10.5% 

Audio 

3.34 

4 

2.3% 

16.3% 

34.9% 

38.4% 

8.1% 

Video 

3.40 

3 

0.0% 

17.4% 

34.9% 

38.4% 

9.3% 

Web-link 

3.33 

5 

1.2% 

18.6% 

34.9% 

37.2% 

8.1% 


Note. SA: strongly agree; A: agree; U: undecided; D: disagree; SD: strongly disagree. 


The learning satisfaction part mainly explored participants’ interest, their satisfaction with their learning 
achievements, and their satisfaction with the interaction in terms of the five annotation functions of HME (text, 
graphic, audio, video, and web-link annotation). Most of the participants thought that the “graphic” (M= 3.53), 
“text” (M=3.45), and “video” (M=3.44) annotations used in this class did increase their interest in the reading 
materials. They also experienced more happiness in their learning. 

For improving learners’ reading achievements and the interaction between learners and the content of the 
materials, materials with “text”, “video”, and “graphic” annotations were considered useful. Text annotation was 
the most popular. The ease of use of text, graphic, and video annotation may be the reason for the result that they 
were the top three popular kinds of annotation while reading online. On the other hand, audio annotation was 
considered the least useful. 





Table 11: 

Analysis of Learning Satisfaction 



Annotation 

M 

Rank 

SD 

D 

U 

A 

SA 

Function 



1 

2 

3 

4 

5 

Q (26.29.32.35.38): Compared with those online materials that did not provide annotation mechanisms, materials 

with annotation used 

in this class did increase my interest 

in the materials. I experienced more 

happiness in 

learning. 








Text 

3.45 

2 

1.2% 

9.3% 

40.7% 

40.7% 

8.1% 

Graphic 

3.53 

1 

0.0% 

10.5% 

36.0% 

43.0% 

10.5% 

Audio 

3.33 

5 

1.2% 

16.3% 

39.5% 

34.9% 

8.1% 

Video 

3.44 

3 

0.0% 

10.5% 

41.9% 

40.7% 

7.0% 

Web-link 

3.37 

4 

0.0% 

14.0% 

40.7% 

39.5% 

5.8% 


Q (27.30.33.36.39): Compared with those online materials that did not provide annotation mechanisms, materials 
with annotation used in this class did improve my learning achievements. 


Text 

3.53 

1 

0% 

9.3% 

34.9% 

48.8% 

7.0% 

Graphic 

3.43 

3 

0% 

12.8% 

41.9% 

34.9% 

10.5% 

Audio 

3.31 

5 

1.2% 

12.8% 

44.2% 

37.2% 

4.7% 

Video 

3.42 

2 

0% 

10.5% 

43.0% 

40.7% 

5.8% 

Web-link 

3.41 

4 

0% 

11.6% 

44.2% 

36.0% 

8.1% 

Q (28.31.34.37.40): A well-known 

saying is 

“Instruction is an interactive process among students, teachers and 

learning materials”. The annotation used in 
content of the materials. 

this class did increase 

the interaction between the learners and the 

Text 

3.52 

1 

0.0% 

10.5% 

36.0% 

44.2% 

9.3% 

Graphic 

3.45 

2 

0.0% 

11.6% 

39.5% 

40.7% 

8.1% 

Audio 

3.37 

5 

0.0% 

11.6% 

48.8% 

30.2% 

9.3% 

Video 

3.43 

3 

1.2% 

9.3% 

41.9% 

40.7% 

7.0% 

Web-link 

3.38 

4 

0.0% 

14.0% 

43.0% 

33.7% 

9.3% 

Note. SA: strongly agree; 

A: agree; U: undecided; D: disagree; SD: strongly disagree. 




According to the participants’ responses, text annotation was regarded as the most helpful and useful annotation 
type, while graphic annotation was regarded as the easiest to use. However, web-link annotation was regarded as 
the least helpful and the least useful. The learners also felt that audio annotation was effective and useful for 
reading comprehension, but was not easy to use. In sum, the exploration of the attitudes of learners showed that 
text annotation was thought to be the most useful type and that it can increase the interaction among learners and 
materials, although graphic annotation was considered the easiest to use, and learners felt interested in making 
graphic annotations because they got more happiness from it. 
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DISCUSSION 

Learning by doing is better than learning by viewing for most types of learners 

Since both Q1 and Q2 were to explore the effectiveness of using different multimedia annotations among 
different characteristics of learners, the researchers dealt the results of these two research questions together in 
this section. The examination of different types of annotation used by the learners revealed that the learner- 
generated annotation group performed better than the instructor-provided annotation group for both cognitive 
style learners and for higher-level learners. This result supports the findings of Pan (2006) and Sung (2007) who 
pointed out that learner-generated multimedia annotation can make a great contribution to language learning. 
Moreover, Dollon and Gabbar (1998) and Ariew and Ercetin (2004) claimed that higher-level learners perfonn 
better when they use learner-generated annotation. 

The results conformed to the activity theory which claims that deep learning occurs when learners are 
encouraged to engage in productive learning activities, and generating learner-generated annotation encourages 
generative processing. It is the learning by doing rationale that claims that learners must engage in an activity 
that is related to the instructional objective. In order to comprehend the text, learners in the learner-generated 
annotation group were asked to surf the Internet and find some related infonnation to make different types of 
annotations by themselves, including text, graphic, audio, video, and web-link annotations. Thus, they had more 
opportunities to learn by themselves than those who read the texts with instructor-provided annotations. 

One of the reasons may be that their high linguistic competence might have enabled the higher-level learners to 
use good reading strategies to promote their reading comprehension of the text in a multimedia environment 
(Ariew & Ercetin, 2004). Furthermore, the learner-generated annotation provided the higher-level learners with a 
better situation to develop their learning autonomy. Besides, higher-level learners have their preferred learning 
strategy uses and would not rely on instructor-provided annotations for reading comprehension, and they might 
be distracted by much less important information provided by instructor-provided annotations while reading 
texts. Learner-generated annotation reinforces learner autonomy because learners have opportunities to choose 
what they think is useful for their learning. While using the annotation system, in order to enhance their reading 
comprehension, learners can use different kinds of annotation, such as text, graphic, audio, video, and web-link 
annotation to help them remember the content or the significant points of the reading materials (Roby, 1999). 

However, learners in the instructor-provided annotation group were asked to view the different types of 
annotation provided by instructors to comprehend the texts, so they did not learn by making their own 
annotations. They just needed to view the annotations they liked. According to the activity theory, learner- 
generated annotation encourages generative processing. However, instructor-provided annotation does not. The 
activity theory predicts that learners who read a text in which they generate their own annotations will perfonn 
better on reading quizzes than those who read the text with instructor-provided annotations. Therefore, the 
results of this research question comply with the rationale that learning by doing is better than learning by 
viewing. 

Learners have positive attitudes toward multimedia annotation use 

The learners thought that the “text” and “audio” annotation types were particularly useful for online reading. 
Especially, “text” annotation was regarded as an effective tool for reading performance. Besides, during the 
reading process, the participants thought that “text” annotation did improve their reading achievement and 
enhance the interaction between the learners and the content of the reading materials. The results also conformed 
to Hwang, Wang, and Sharpies (2007), Pan (2006), and Sung (2007) who have all claimed that learners feel that 
by using “text” annotation and other functions of annotation systems, like graphic, audio, and video annotations, 
it becomes easier for them to review the content of the learning materials. The results also support those of Chun 
(2001) and Mayer (2001) who have claimed that extraneous cognitive load can be decreased by presenting 
reading material accompanied with words and pictures instead of only in words. Thus, one of the reasons may be 
that “text” annotation was easy to use, and it can be used without training. Another reason is that there was no 
cognitive overload because the dual coding theory maintains that learning can be facilitated when materials that 
involve both verbal and visual systems are utilized simultaneously. The results of Chen’s (2009) study, which 
indicated that learners using video annotation perform significantly better than those using picture annotation on 
tests, also conformed to the argument that viewing various kinds of annotation encourages generative processing. 

However, not all types of annotation facilitate reading comprehension (Akbulut, 2008). Web-link annotation and 
audio annotation were regarded as the two least helpful and least useful annotation types by the students in this 
study. These findings partially comply with the findings of Ariew and Ercetin (2004) and Saker and Ercetin 
(2005), who found that audio annotation might distract readers, and that it has a negative impact on reading 
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comprehension. As for cognitive theory, one of the reasons may be that presenting too many elements to be 
processed may lead to cognitive overload. Another reason may be that viewing various types of annotation 
causes cognitive overload. 

Another reason may be related to the difficulty of using audio annotation, because learners need to download and 
implement other software to record their annotations. Thus, some of the learners may feel that it is troublesome, 
and thus it may have decreased their willingness to use audio annotation while reading in the multimedia 
environment. In addition, as for web-link annotation, although it is easy to use, the content and information it 
provides may not have direct benefits for reading comprehension. Thus, the learners in this study seldom used 
this kind of annotation to help them comprehend the text. 

CONCLUSION 

Students scored higher on a post English proficiency test after reading articles in which they were asked to 
construct multimedia annotation (learner-generated) that after reading articles that contained multimedia 
annotation (instructor-provided). The effect sizes favoring the both cognitive style learners and higher-level 
learners were large (d > 1 for FI, FD, and higher-level learners) when they received the learner-generated 
annotation treatment, and was small (d = .16) for the lower-level ones. The low-lever learners did not differ 
significantly on the proficiency test, which indicates that two groups (learner-generated and instructor-provided) 
of lower-level learners reached the same level of learning the basic materials. Besides the lower-level learners, 
the learner-generated groups showed overall deeper understanding as compared to the instructor-provided 
groups. The pattern of results is consistent with activity theory, which predicts that students learn more deeply by 
doing than by viewing. 

The researchers do not recommend asking lower-level learners to construct their own multimedia annotation, 
especially when the learning task is difficult. Generating annotation may increases the level of extraneous 
cognitive processing that limits generative processing, especially when the information is new and techniques are 
unfamiliar. While learners studying simple material or studying complex materials with unlimited time, 
generating annotation by themselves is a better option to promote organization and integration (Stull & Mayer, 
2007). 

From the results of annotation attitude questionnaire, the analysis show that most of the students have positive 
attitudes toward the questions for all dimensions (perceived usefulness, perceived ease of use, and learning 
satisfaction). These results reveal the research shows great potential of an annotation mechanism to enhance 
online reading. This study is limited in several areas that should be investigated in future studies. First, the 
sample of field-dependent participants (N = 36) was much less than field-independent participants (N = 57), 
which may decrease the statistical power for analyses. Future research should choose more FD observations to 
include in a statistical sample. 

On the other hand, the English reading proficiency test and annotation attitude questionnaires were administered 
as dependent variables to explore the effects of using different multimedia annotations quantitatively. The 
content analysis of annotation behaviors and student annotation interaction are suggested to discover annotation 
behaviors for future examination. These quality data might be helpful for instructors to understand student 
thinking process and learning difficulty through the analysis. 
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